Early Detection of Meningitis Outbreaks: Application of Limited-baseline Data.

Background
There is no published study evaluating the performance of cumulative sum (CUSUM) algorithm on meningitis data with limited baseline period. This study aimed to evaluate the CUSUM performance in timely detection of 707 semi-synthetic outbreak days.


Methods
Simulated outbreaks were generated using syndromic data on fever and neurological symptoms from Mar 2010 to Mar 2013 in Hamadan Province, the west of Iran. The performance of CUSUM algorithms, numbered from 1 to 11, in timely detection of outbreaks was measured using sensitivity, specificity, false alarm rate, likelihood ratios and area under the receiver operating characteristics (ROC) curve.


Results
The highest amount of sensitivity was related to algorithm11 (CUSUM(3-9 D11)) and it was 52% (95% CI: 49%, 56%). Minimum amount of false alarm rate was related to CUSUM(1-7 D5) algorithm equal to 8% (95% CI: 5, 10) and the best amount of positive likelihood ratio was related to CUSUM(1-7 D4) equal to 4.97. CUSUM(1-7 D1) has the best performance with AUC curve equal to 73% (95 CI%: 70%, 76%), as well.


Conclusion
The used approach in this study can be the basis for applying CUSUM algorithm in conditions that there is no access to recorded baseline data about under surveillance diseases or health events.


Introduction
Despite recent advances in the diagnosis of infectious diseases and the development of programs of vaccination against meningococcal meningitis, acute meningitis is still an important cause of illness and a serious threat to mankind and its related diseases annually causes 170,000 deaths in the world (1,2). In Iran, meningitis is a notifiable disease and suspected cases of this disease i.e. syndromic data on fever and neurological symptoms are reported and registered daily in national surveillance system for communicable diseases (3).
Early detection and timely response to occurred outbreaks is a priority for public health authorities (4,5). Generally, conventional surveillance systems respond to outbreaks based on laboratory time-consuming diagnosis. A new type of public health surveillance systems have special potential for rapid and timely detection of outbreak that known as syndromic surveillance system (6). Syndromic surveillance systems used different data sources and methods to detect outbreaks or unusual increases of health events (7,8). One of the known algorithms for early detection of outbreaks is Cumulative Sum (CUSUM) algorithm, which is able to detect small changes in the process mean that control charts more quickly (9). CUSUM is under the umbrella of statistical process control-based methods (6), and it is used especially when historical data isn't available or existing data are incorrect (10)(11)(12). Outbreak detection algorithms methods typically require historical baseline data for a long period such as about 3-5 yr ago to be applied practically (13). Public health surveillance systems in order to identify outbreaks in shorter time as soon as possible have access to short term baseline data and less than three years (14). Consider to the lack of access to long period baseline data for majority of communicable diseases or newly emerging health events in many countries. It is necessary to develop applied algorithms for such situation. The necessity of required limited baseline data algorithms is more tangible while staff of public health surveillance systems faced to similar circumstances like SARS epidemic. Accordingly, this study aimed to evaluate the performance of CUSUM algorithm in timely detection of meningitis outbreak without longterm baseline data (Limited baseline data) based on semi synthesis approach in Hamadan Province.

Data
The performance of CUSUM algorithms was approached on syndromic data on fever and neurological symptoms (suspected cases of meningitis) from Mar 2010 to Mar 2013 in Hamadan Province, western Iran. We enrolled aggregate data of 1506 cases with sudden onset of fever above 38 °C and a clinical sign or symptoms such as neck stiffness, loss of consciousness, headache, vomiting and sudden neurological complications, reported daily in national surveillance system for communicable diseases in the province. To evaluate the performance of the algorithm, simulated outbreaks have been injected to pre-processed data of suspected cases of meningitis. Details on methodology of data preprocessing and removing explainable patterns have been described elsewhere (15).

Outbreak Simulation
Different types of meningitis related epidemic curves, consider to possible size, duration and shape, were generated. In case of size, we injected one to eight more cases to report suspected cases of meningitis during study period. The corresponding time duration of epidemic curve was 7, 14 and 21 d. Shapes of simulated outbreaks had uniform, exponential and linear distribution. Of 55 simulated epidemic curves, 23 outbreaks were 7 d period (15 uniform, 4 exponential and 4 linear), 18 outbreaks were 14 d period (10 uniform, 4 exponential and 4 linear) and other outbreaks were 21 d (8 uniform, 3 exponential and 3 linear). Finally, concerning epidemiological profile of meningitis and dynamics of disease transmission, 707 outbreak days were injected to 1085 d of real data on meningitis as known in literature as semisynthetic outbreak. Accordingly, of 1085 d, 378 d were without outbreak. Fig. 1 depicts the size and duration of both injected cases on baseline cases and simulated outbreaks.

Outbreak Detection Algorithm
We used CUSUM algorithms to detect outbreaks as follows. In this study, 11 CUSUM algorithms was evaluated based on limited data from the previous 7 d in closest proximity to the current value(days t-1 through t-7 ), or based on limited data from the past 7 d with an interval of two days, (days t-3 through t-9 ) with different threshold levels of meningitis outbreaks. The baseline period is always chosen from one week ago or each week attributed to the current value. CUSUM equation can be written as follows (14): Where, Y t is number of reported suspected cases of meningitis on day t (t = 1, 2... n), CUSUM t-1 is the value of CUSUM on day t-1 and σ is the standard deviation of observed data during the interesting week.

Results
During the study period of applied data, 1506 suspected cases of meningitis were reported and included as well. Details on descriptive statistics for fever and neurological symptom syndrome i.e. suspected cases of meningitis from Mar 2010 to Mar 2013 as depicted in Fig. 2 in Hamadan Province were described elsewhere (16,17). The highest amount of sensitivity was related to CUSUM (3-9 D11) which equals to 52% (95% CI:49%, 56%). The highest value of specificity was related to CUSUM (1-7D5) and equals to 92% (95% CI:90%, 95%). The minimum amount of false alarm rate was related to CUSUM (1-7D 5) and equals to 8% (95% CI: 5%, 10%) and minimum amount of false negative rate was related to CUSUM (3-9 D 11) . The best amount of positive likelihood ratio was related to CUSUM (1-7 D 4) and equals to 4.97, and minimum negative likelihood ratio was related to CUSUM (1-7 D 1) . More details on measures of algorithms' performance including sensitivity, specificity, false alarm rate, false negative rate, positive and negative likelihood ratios analyses are shown in Table 2. The CUSUM (1-7 D 1), algorithm 1, with the ROC value 73% (95% CI: 70, 76) has shown the best performance in comparison with other algorithms (Fig. 3). Fig. 4 shows the ROC values for different CUSUM algorithms with baseline period during 3-9 past days. Corresponding values for timeliness index according to shape of simulated outbreaks are shown in Table 3.
Minimum and maximum time interval between the actual day of outbreak and the time of alarm by algorithm were observed in CUSUM (3-9 D 11) with 5.94 d and in CUSUM (1-7 D 5) with 7.17 d, on average. Moreover, simulated outbreaks with exponential distribution have worked timely in comparison with uniform and linear distributions. So the best performance in timely detection of outbreaks was related to CUSUM (3-9 D9) .

Discussion
Since any algorithm has not ability to detect all outbreaks effectively and evaluate the performance of algorithms provides useful and important information regarding strengths and weaknesses of applied algorithm, this study was designed to evaluate the performance of CUSUM algorithms with limited baseline period and to identify timely detection of meningitis.
The results of this study as well as some studies aimed to evaluate the performance of the CUSUM algorithm in different circumstances, using semi synthesis and synthesis approach has good performance in detecting semi-synthetic outbreaks (18). Our findings are consistent with the results of another study (19), was aimed to evaluate the CUSUM performance to detect small deviations from the mean. The best performance in terms of sensitivity was observed for CUSUM (3-9 D 11) with an average data on 3-9 d ago. This result is consistent with the results of Hut Wagner et al. (14). However, our findings regarding false positive rate are in contrast to related study (14). This disagreement could be justified while considering the role data pre-processing in our study instead of raw data with explainable patterns. Simulated data with regard to comprehensive characterization of the properties of outbreaks provides reliable information to compare outbreak detection methods. Application of simulated outbreaks with different size, shape and time periods and adding simulated outbreaks to preprocessed data of suspected cases of meningitis are main strengths of this work. Strength of this study is application of preprocessed values of LOWESS method (Details are described elsewhere as mentioned in methods) instead of the raw of suspected cases of meningitis, which reduces the false alarm rate, increase sensitivity, and timely detection of outbreak occurs. The main limitations of the present study are (a) simulated outbreaks have been generated consider to knowledge of the dynamics of meningitis in Hamadan Province and it is different from other data, thus it is limited to compare its results with other studies. (b) Applied algorithms are evaluated under near real conditions in the era of possible outbreaks, not real conditions. To resolve this problem we state comprehensive information about added outbreaks, that including size, shape and period of potential outbreaks. However, we have not accessed to official data regarding any outbreak occurred during study period to evaluate algorithms using actual data source as gold standard (4).

Conclusion
The used approach in this study could be the basis for applying CUSUM algorithm in conditions that there is no access to recorded baseline data on interested diseases or health events. Nevertheless, the simultaneous using of other methods of outbreak detection is recommended. Methodol-ogy of this work is not limited to meningitis and can be applied to other syndromic data in public health surveillance.

Ethical considerations
Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.